Speech-Based Situational Awareness for Crisis Response

I. INTRODUCTION The goal of our work is to explore research in the framework of an end-to-end speech processing system that can automatically process human conversations to create situational awareness during crisis response. Situational awareness refers to knowledge about the unfolding crisis event, the needs, the resources, and the context. Accurate assessment of the situation is vital to enable first responders (and the public) to take appropriate actions that can have significant impact on life and property. Consider, for instance, a situation of a large structural fire wherein teams of fire fighters enter into a burning building for search and rescue. Knowledge of the location of fire fighters, their physiological status, the ambient conditions and environment are critical for the success and safety of both the victims and fire fighters. Appropriate situational awareness is critical not just at incident level, but at all levels of response. For instance, knowledge of occupancy levels, the special needs of the populace, the road-closures, the geographical scope of the disaster (e.g., the fire perimeter), etc. play a vital role in evacuation and shelter planning and in organizing medical triage. The importance of accurate and actionable situational awareness in crisis response is now well recognized [1] and has led to significant research on appropriate sensing, networking, sensor processing, information sharing, data management, and decision support tools. The research team at the UC Irvine, Center for Emergency Response Technologies (CERT) has been extensively involved in working with a variety of first responder organizations to build a variety of such tools in the context of the NSF funded RESCUE (Responding to Crises and Unexpected Events) project [2] and the DHS funded SAFIRE project [3]. Our experience working with rescue personnel has clearly established that while sensors (including motes, video, physiological, location, environmental) are important, speech is undoubtedly the single most important source of situational information. The very first point of contact of citizens with the responders during an emergency is through a telephone call to the 911 dispatch system. In case of large disasters that involve a larger team of responders (such as a fire-fighting team), the primary mechanism used for communication and coordination among response teams is through radios carried by the first responders. Such conversations contain perhaps what